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Foreword 


This  work  is  in  support  of  a  multi-year  project  to  evaluate  “people-centered”  aspects 
of  computer-based  training  (CBT)  or  technology  assisted  instruction.  One  component  of 
this  project  will  identify  the  prior  experience  new  recruits  have  with  technology  assisted 
instruction  and  computer-based  training.  The  core  component  of  the  work  will  be 
developing  a  longitudinal  database  that  will  include  individual  difference1  measures  of 
non-cognitive  characteristics  (e.g.,  personality,  temperament,  interests)  of  students, 
student  engagement  and  satisfaction  with  instruction,  student  outcomes,  and  early  job 
satisfaction  and  performance.  Initial  technical  training  pipelines  will  be  chosen  that  vary 
in  length,  complexity,  and  instructional  methods  (e.g.,  primarily  CBT,  instructor-led, 
and  blended  methods)  to  evaluate  interactions  between  student  non-cognitive 
characteristics  and  learning  outcomes.  This  report  is  a  selected  literature  review  to 
provide  a  starting  point  from  which  to  view  the  interaction  between  individual 
differences  in  non-cognitive  characteristics,  instructional  methods,  and  learning 
outcomes  to  inform  decisions  on  which  non-cognitive  characteristics  to  use  in  the 
research.  While  the  results  were  not  predetermined,  following  the  review  of  critical  non- 
cognitive  characteristics,  the  recommendation  is  to  utilize  a  Navy-developed  battery  of 
personality  traits  referred  to  as  the  Navy  Computer  Adaptive  Personality  Scales 
(NCAPS)  because  it  covers  the  characteristics  that  should  be  measured,  the  battery  is 
owned  by  the  Navy  and  is  not  proprietary  (avoiding  short  and  long  term  costs),  the  test 
has  proven  to  be  fake  resistant,  has  been  evaluated  on  over  25,000  Navy  personnel,  and 
it  is  web-enabled  which  reduces  barriers  to  administration. 

This  research  was  sponsored  by  the  Naval  Education  and  Training  Command 
(NETC),  funded  by  the  Office  of  Naval  Research  (ONR-34),  and  executed  by  Navy 
Personnel  Research,  Studies,  and  Technology  (NPRST)  department  of  the  Bureau  of 
Naval  Personnel  (BUPERS-i). 


David  M.  Cashbaugh 
Director 


1  The  term  “individual  differences”  refers  to  the  field  of  study  called  differential  psychology.  This  field 
focuses  on  how  individuals  (or  groups  of  individuals)  differ  from  one  another;  for  example,  in  height, 
simple  reaction  time,  auditory  acuity,  topical  knowledge,  intelligence,  personality  traits,  interests,  and 
attitudes.  Typically,  some  measurement  of  a  construct  is  developed  (e.g.,  a  mechanical  knowledge  test), 
administered  to  people,  and  the  dispersion  of  scores  (“differentials”)  is  observed  and  scrutinized. 
“Individual  differences”  then  refers  to  measures  that  differentiate  one  individual  from  another  on  a 
specific  measure.  The  subfield  of  individual  differences  that  focuses  on  the  science  of  measurement 
properties  that  underlie  formal  tests  (e.g.,  SAT,  ACT)  is  called  psychometrics.  In  the  current  context, 
“individual  differences”  will  refer  to  measures  of  constructs  that  differentiate  (or  scale)  people  relative  to 
one  another. 
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Overview 


The  purpose  of  this  literature  review  is  to  identify  individual  differences  (e.g., 
motivation,  personality  traits,  self-concept,  interests,  etc.)  that  are  likely  to  predict  and 
explain  training  performance  in  Navy  classrooms  and  which  may  interact  with  various 
instructional  methods  (e.g.,  self-paced,  computer-based  training  [CBT],  instructor-led, 
computer-assisted  and  instructor-led  blended  training,  etc.).  Training  theory  and 
research  widely  recognizes  that  motivation  to  learn  has  a  direct  effect  on  learning 
outcomes.  In  addition,  individual  characteristics  and  situational  factors  are  also 
recognized  as  having  direct  and  indirect  effects  on  motivation  to  learn  and  learning 
outcomes  (Colquitt,  LePine,  &  Noe,  2000).  Not  only  will  individual  characteristics  have 
a  direct  affect  on  learning  performance  and  outcome  measures,  but  they  can  interact 
with  the  delivery  method  of  training  to  change  the  way  trainees  respond  to  training 
(Gully  &  Chen,  2010;  Salas,  Kosarzycki,  Burke,  Fiore,  &  Stone,  2002). 


How  I  ndividual  Differences  I  mpact  Training  Outcomes 


Gully  and  Chen  (2010)  suggest  that  individual  differences  have  an  impact  on 
learning  performance  through  four  intervening  mechanisms:  (a)  information-processing 
capacity,  (b)  attentional  focus  and  metacognitive  processing,  (c)  motivation  and  effort 
allocation,  and  (d)  emotional  regulation  and  control.  Information-processing  capacity 
includes  general  cognitive  ability,  fluid  and  crystallized  intelligence,2 3  and  working 
memorys  capacity.  Attentional  focus  and  metacognitive  processing  includes  cognitive 
resources  related  to  the  learning  task  or  to  the  situation.  Metacognition  has  been 
described  as  “thinking  about  your  thinking.”  These  abilities  are  required  to  develop 
plans  and  to  evaluate  progress,  and  therefore,  are  critical  for  learning  to  occur  (Flavell, 
1979).  Motivation  and  effort  allocation  includes  general  motivation  (motivation  to 
learn),  as  well  as  more  specific  motives  such  as  learning  goal  orientation.  Emotional 
regulation  and  control  refers  to  the  processes  involved  in  controlling  negative  emotional 
reactions  (e.g.,  anxiety  and  frustration)  and  the  generation  of  positive  facilitative 
emotions  during  training. 

Many  of  Gully  and  Chen’s  (2010)  intervening  mechanisms  can  be  considered  as  part 
of  the  self-regulatory  system.  Self-regulation  is  a  collection  of  abilities  and  processes 
that  help  individuals  to  stay  focused  on  a  task,  to  monitor  their  own  progress  in 
completing  that  task,  and  to  control  their  thoughts,  feelings,  and  behaviors.  Self¬ 
regulation  also  includes  metacognitive  abilities,  including  memory,  attention,  and  goal 


2  Fluid  and  crystallized  intelligence  (abbreviated  Gf  and  Gc,  respectively)  are  facets  of  general  intelligence 
originally  identified  by  Raymond  Cattell  (1971).  Fluid  intelligence  or  fluid  reasoning  is  the  capacity  to 
think  logically  and  solve  problems  in  novel  situations,  independent  of  acquired  knowledge.  Crystallized 
intelligence  is  the  ability  to  use  skills,  knowledge,  and  experience;  it  should  not  be  equated  with  memory 
or  knowledge,  but  it  does  rely  on  accessing  information  from  long-term  memory. 

3  Working  memory  is  the  limited  capacity  ability  to  actively  hold  information  in  consciousness  that  is 
needed  to  perform  complex  tasks  such  as  reasoning,  comprehension  and  learning.  Working  memory  tasks 
are  those  that  require  the  goal  orientated  active  monitoring  or  manipulation  of  information. 
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setting  for  problem  solution.  This  enables  learners  to  make  the  best  use  of  their 
knowledge,  time,  and  skills  (Pressley,  1995). 

Self-regulatory  theories  have  proven  relevant  to  various  lines  of  research,  including 
metacognition,  achievement  goals,  intrinsic  motivation,  action  control,  appraisal  (or 
evaluation)  processes,  autonomy  and  self-determination  in  goal-setting,  and  cognitive 
or  metacognitive  strategic  use  in  the  implementation  of  goals  (Efklides,  Niemivirta,  & 
Yamauchi,  2002),  including  learning  (Bandura,  1991;  Bell  &  Kozlowski,  2002a; 
Boekaerts,  Pintrich,  &  Zeidner,  2000;  Flavell,  1979;  Keith  &  Frese,  2005;  Kraiger  & 
Jerden,  2007;  Santhanam,  Sasidharan,  &  Webster,  2008;  Sitzman,  Bell,  DeRouin, 
Fritzsche,  &  Salas,  2005;  Sitzmann,  Bell,  Kraiger,  &  Kanar,  2009;  Sitzmann,  Kraiger, 
Stewart,  &  Wisher,  2006).  Self-regulated  learning  models  attempt  to  integrate  cognitive, 
affective-motivational,  and  behavioral  aspects  of  learning  (Boekaerts  et  al.,  2000).  Self- 
regulated  learning  models  are  advantageous  in  learning  research  because  they  allow 
researchers  to  describe  the  many  levels  and  processes  involved  in  successful  learning 
outcomes,  to  explain  reciprocal  and  recurrent  relationships  established  between  these 
components,  and  to  directly  relate  learning  with  goals,  motivation,  and  emotions 
(Boekaerts,  1999). 

Self-regulated  learning  includes  processes  such  as: 
setting  goals  for  learning,  attending  to  and  concentrating  on  instruction, 
using  effective  strategies  to  organise,  code  and  rehearse  information  to  be 
remembered,  establishing  a  productive  work  environment,  using  resources 
effectively,  monitoring  performance,  managing  time  effectively,  seeking 
assistance  when  needed,  holding  positive  beliefs  about  one’s  capabilities, 
the  value  of  learning,  the  factors  influencing  learning  and  the  anticipated 
outcomes  of  actions,  and  experiencing  pride  and  satisfaction  with  one’s 
efforts  (Schunk  &  Ertmer,  2000,  pp.  631). 

In  order  to  self-regulate,  trainees  must  engage  in  both  emotional  control  and 
metacognition  (Kanfer,  1996;  Kanfer  &  Ackerman,  1996),  both  of  which  have  direct 
effects  on  learning  (Keith  &  Frese,  2005).  Emotional  control  limits  the  impact  of 
negative  affect  (e.g.,  anxiety  or  worry)  during  learning  while  metacognitive  ability 
involves  controlling  one’s  cognitions,  planning,  monitoring,  and  evaluating  one’s 
progress  during  task  execution  (Kanfer  &  Ackerman,  1989).  There  has  not  been  as  much 
research  on  other  self-regulation  processes  (e.g.,  motivational  and  emotional  regulation) 
on  learning  as  there  has  been  on  metacognitive  ability,  but  researchers  have  recently 
taken  note  and  begun  to  recognize  their  impact  (Pintrich,  2000). 

Metacognitive  abilities  are  involved  in  decisions  about  how  personal  resources  are 
allocated  during  training,  in  monitoring  and  controlling  resources  to  keep  them  on  on- 
task  activities  rather  than  off-task  activities,  and  in  making  changes  in  resource  and 
effort  allocations  during  training  (Kanfer  &  Ackerman,  1989).  Often  deficiencies  in 
performance  can  be  explained  by  inaccurate  self-assessments  of  current  knowledge 
levels,  deficits  in  the  amount  of  effect  allocated  to  the  task,  or  poor  decisions  during 
training  (Brown,  2001;  Kanfer  &  Ackerman,  1989;  Sitzmann,  Ely,  Brown,  &  Bauer, 
2010).  Social-cognitive  models  of  self-regulated  learning  include  self-concepts, 
motivational  feelings  and  beliefs,  as  well  as  learning  strategies  and  metacognitive  skills 
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(i.e.,  the  knowledge  and  control  that  individuals  have  over  their  own  thinking  processes) 
(Zimmerman,  1995). 

Because  computer-based  training  (CBT)  tends  to  provide  trainees  with  more 
opportunities  to  have  control  over  their  learning  experience  than  traditional  classroom 
instruction  (Sitzmann  et  al.,  2006),  using  self-regulation  theories  will  prove  useful  as  an 
organizational  framework  for  looking  at  how  individual  differences  influence  CBT 
learning  outcomes  and  success.  Indeed,  research  suggests  that  trainees’  poor  ability  to 
self-regulate  is  why  some  trainees  fail  to  complete  or  succeed  in  training  when  using 
CBT  (Bell  &  Kozlowski,  2002a;  DeRouin,  Fritzsche,  &  Salas,  2005;  Kraiger  &  Jerden, 
2007). 


I  ndividual  Differences  that  Affect  Training  Outcomes 


Metacognitive  Abilities  (MCA) 

Individual  differences  can  affect  learning  performance  in  MCA;  fortunately,  MCA  is 
itself  an  individual  difference  construct  that  is  often  measured.  While  metacognitive 
knowledge  is  comprised  of  several  components,  the  research  and  theoretical  focus  on 
MCA  typically  focuses  on  two  components,  monitoring  and  control.  Metacognitive 
monitoring  skills  include  evaluating  learning  progress  and  predicting  outcomes, 
whereas  metacognitive  control  includes  decisions  on  where  to  allocate  effort  and 
resources.  Some  models  of  self-regulated  learning  have  posited  metacognitive 
monitoring  as  the  key  to  self-regulated  learning  (Butler  &  Winne,  1995;  Winne,  1996, 
1997).  Self-monitoring  skills  contribute  to  the  ability  to  know  how  well  one  is 
performing,  when  one  is  likely  to  be  accurate  in  judgment,  and  when  one  is  likely  to  be 
in  error  (Chi,  Glaser,  &  Rees,  1982).  These  skills  are  essentially  declarative  knowledge4 
about  the  interactions  between  person,  task,  and  strategic  characteristics  (Flavell,  1979). 
Metacognitive  knowledge  refers  to  procedural  knowledges  for  regulating  problem 
solving  and  learning  activities,  which  can  be  correct  or  incorrect  (Brown  &  DeLoache, 
1978;  Veenman  &  Spaans,  2005). 

Some  researchers  consider  self-regulation  to  be  a  precursor  to  metacognition  (e.g., 
Brown  &  DeLoache,  1978;  Kluwe,  1987),  whereas  others  regard  metacognition  as  a 
precursor  to  self-regulation  (e.g.,  Winne,  1996;  Zimmerman,  1995).  According  to  the 
latter  social-cognitive  perspective,  self-regulation  also  involves  motivational  and  social- 
emotional  processes.  Of  particular  interest,  the  majority  of  research  looking  at  the 
influence  of  MCA  on  learning  performance  has  been  conducted  in  CBT  environments 
(Gully  &  Chen,  2010).  Learning  performance  in  these  situations  is  considered  to  be 


4  “Declarative  knowledge”  is  factual,  descriptive,  or  propositional  information.  For  example,  knowledge  of 
what  a  bicycle  is  (in  contrast  to  knowledge  of  how  to  ride  a  bike).  Shorthand  mnemonic:  knowing  what 
(vs.  knowing  how). 

5  “Procedural  knowledge”  is  process  or  procedural  information.  For  example,  knowledge  of  riding  a 
bicycle  (in  contrast  to  knowledge  of  what  a  bicycle  is).  Shorthand  mnemonic:  knowing  how  (vs.  knowing 
what). 
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more  affected  by  MCA  because  the  absence  of  a  face-to-face  instructor  increases  the 
need  for  individual  learner  control. 

MCA  is  a  component  in  several  models  of  intelligence.  Brown  (1987)  saw  conscience 
MCA  as  a  characteristic  of  high  intelligence.  Sternberg’s  Triarchic  Theory  of  Human 
Intelligence  refers  to  “metacomponents”  that  control  other  cognitive  components  as  well 
as  receive  feedback  from  these  components  in  a  feedback  loop  necessary  for  learning 
(Sternberg,  1984, 1986a,  1986b).  However,  Sternberg  (1990)  later  maintained  that 
metacognitive  skills  cannot  be  equated  with  intellectual  ability.  Many  studies  have 
shown  that  metacognitive  skills,  although  moderately  correlated  to  intelligence, 
contribute  to  learning  performance  on  top  of  intellectual  ability.  For  instance, 
metacognitive  skills  outweighed  intelligence  in  predicting  learning  performance  in  one 
study  of  secondary  students  (Veenman  &  Spaans,  2005)  but  intellectual  ability  was  still 
critical.  In  another  study,  intellectual  ability  uniquely  accounted  for  10  percent  of 
variance  in  learning  on  average,  while  metacognitive  skills  uniquely  accounted  for  17 
percent  of  variance  in  learning,  and  both  predictors  shared  another  20  percent  of 
variance  in  learning  (Veenman,  Van  Hout-Wolters,  &  Afflerbach,  2006;  Veenman  & 
Spaans,  2005).  The  implication  is  that  an  adequate  level  of  metacognition  may 
compensate  for  trainees’  cognitive  limitations,  at  least  up  to  some  minimal  threshold.6 7 

Measurement  of  MCA  can  be  based  on  actual  behaviors  and  performance  during 
training,  think-aloud  reports  in  which  individuals  describe  their  thinking  while  it  is 
occurring,  retrospective  interviews  in  which  people  recall  what  they  had  been  thinking, 
or  self-report  measures.  State  and  trait?  versions  of  the  self-report  measures  have  been 
used  to  measuring  MCA,  with  trait  MCA  influencing  state  MCA  (Goos,  Galbraith,  & 
Renshaw,  2000;  Hong  &  O’Neil,  2001;  O’Neil  &  Abedi,  1996;  Schraw  &  Dennison,  1994; 
Tobias  &  Everson,  1997). 

Metacognitive  skills  initially  develop  in  separate  domains  of  knowledge,  and  later 
become  generalized  across  domains  (Veenman  &  Spaans,  2005).  MCA  may  be  trainable 
and  amenable  to  training  design  factors  (Bell  &  Koslowski,  2002a,  2002b,  2008; 
Dunlosky,  Kubat-Silman,  &  Hertzog,  2003;  Keith  &  Frese,  2005;  Schmidt  &  Ford, 

2003).  Metacognitive  skills  can  be  trained  via  computer-based  training  with 
metacognitive  instructions  before  a  learning  activity  (Schmidt  &  Ford,  2003)  or 
embedded  within  the  learning  activity  (Aleven  &  Koedinger,  2002). 

Motivation 


6  MCA  and  these  results  derive  from  mainstream  psychology  and  education  literatures.  More 
sophisticated  and  subtle  distinctions  are  made  in  the  cognitive  neuropsychological  literature.  Here,  the 
components  of  what  are  referred  to  as  metacognitive  monitoring  and  control,  are  called  executive 
functions  (EF)  but  serve  similar  purposes  in  support  of  learning  and  intelligence.  Examples  of  primary 
executive  functions  include  “inhibiting”  prepotent  responses,  “shifting”  mental  sets,  and  “updating” 
working  memory.  Even  still,  this  literature  on  executive  functions,  with  both  normal  and  frontal  lobe 
damaged  populations,  have  found  that  executive  functions  are  critical  to  learning  and  intelligence. 
Moreover,  the  evidence  is  that  there  is  only  a  modest  degree  of  overlap  between  the  executive  functions. 
See  for  example,  Friedman  et  al.  (2006)  and  Salthouse,  Atkinson,  &  Berish  (2003). 

7  “State”  refers  to  a  transitory  characteristic  of  an  individual  whereas  “trait”  refers  to  an  enduring  response 
proclivity.  For  example,  there  is  “state  anxiety”  which  refers  to  the  anxiety  that  a  person  is  currently 
experiencing  and  is  in  contrast  to  “trait  anxiety”  which  refers  to  the  tendency  of  a  person  to  generally 
experience  anxiety  (or  not). 
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Motivation  to  Learn 


Trainee  characteristics  have  been  found  to  have  a  wide  influence  on  learning  and 
training  outcomes,  mainly  through  their  effect  on  motivation  to  learn  (Thayer  & 
Goldstein,  2010).  Motivation  to  learn  has  been  defined  as  a  malleable  individual 
characteristic  that  affects  the  willingness,  direction,  intensity,  and  persistence  of 
learning-directed  behavior  and  also  influences  the  choices  individuals  make  during 
learning  activities  (Noe,  1986;  Noe  &  Schmitt,  1986).  Several  studies  show  that  it  is 
positively  related  to  learning  and  is,  in  fact,  a  critical  precursor  (Baldwin,  Magjuka,  & 
Loher,  1991;  Campbell,  1989;  Colquitt  et  al,  2000;  Colquitt  &  Simmering,  1998;  Gist, 
Stevens,  &  Bavetta,  1991;  Goldstein,  1993;  Mathieu,  Tannenbaum,  &  Salas,  1992; 
Martocchio  &  Webster,  1992;  Noe,  1986;  Noe  &  Schmitt,  1986;  Quinones,  1995; 
Tannenbaum,  Mathieu,  Salas,  &  Cannon-Bowers,  1991).  Trainees  who  are  motivated  to 
learn  are  more  likely  to  engage  in  learning  activities  they  believe  are  beneficial.  They  pay 
more  attention  during  class,  study  harder,  and  are  more  likely  to  request  help  when  they 
need  it  (Pintrich  &  Garcia,  1991).  The  effects  of  other  individual  characteristics  on 
motivation  to  learn  will  be  referred  to  throughout  this  review. 

I  ntrinsic  Motivation 

Research  by  Deci,  Ryan,  and  associates  (e.g.,  Deci,  Connell,  &  Ryan,  1989;  Deci, 
Koestner,  &  Ryan,  1999;  Deci  &  Moller,  2005)  has  shown  that  motives  can  be  classified 
as  intrinsic  or  extrinsic.  Individuals  are  thought  to  be  internally  motivated  if  they  find 
the  task  rewarding  from  the  experience,  pleasure,  and  satisfaction  inherent  in  the 
activity.  In  contrast,  individuals  are  externally  motivated  if  they  participate  in  the  task  as 
a  means  to  obtain  rewards  that  are  not  directly  part  of  the  task,  for  example,  money  or 
praise.  Compared  to  extrinsically  motivated  trainees,  intrinsically  motivated  trainees 
usually  have  better  training  performance,  enjoy  the  training  more,  are  more  curious 
about  the  topic,  and  engage  in  deeper  levels  of  learning  (Deci  &  Ryan,  2000). 
Intrinsically  motivated  trainees  are  more  likely  to  have  a  learning  goal  orientation 
during  training  (Harackiewicz  &  Elliot,  1993;  Pintrich  &  de  Groot,  1990).  Students  with 
greater  intrinsic  motivation  who  also  view  the  learning  activity  as  important,  are  also 
more  likely  to  use  adaptive  self-regulatory  strategies  (Schunk,  2005) 

Career  I  nterests 

Although  there  is  not  a  lot  of  research  in  the  area  of  the  effects  of  career  interests  on 
training  performance,  Gully  and  Chen  (2010)  point  out  that  individuals  who  are 
interested  in  and  have  a  vocational  aptitude  for  a  particular  career  should  be  more  likely 
to  maintain  motivation  and  attentional  focus  in  training  relevant  to  their  career,  based 
upon  the  observed  overlap  between  personality,  ability,  and  interests  (c.f.,  Ackerman  & 
Heggestad,  1997;  Barak,  2001).  They  expect  that  vocational  interests  will  have  similar 
effects  as  personality  and  other  interests,  although  the  effect  of  interests  may  be  more 
pronounced  when  trainees  are  positively  affected  by  the  training  material  because  they 
should  process  content  more  effectively  and  perhaps  more  deeply  as  well. 
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Goal  Orientation/  Achievement  Goals 

A  goal  orientation  is  a  cognitive  representation  of  a  goal  that  the  trainee  establishes 
during  learning.  Initial  research  on  goal  orientation  (e.g.,  Dweck,  1986;  Eison,  1981; 
Nicholls,  1975)  compared  learning  goal  orientation  (LGO,  also  known  as  mastery  goal 
orientation)  with  performance  goal  orientation  (PGO,  also  known  as  achievement  goal 
orientation).  A  trainee  with  a  learning  goal  orientation  (LGO)  wants  to  increase  his  or 
her  competence  and  research  generally  shows  that  these  trainees  tend  to  learn  more 
from  experimentation  and  failure  since  they  are  testing  their  knowledge  against  self- 
referenced  standards  (e.g.,  doing  better  than  they  did  the  last  time  they  attempted  the 
learning  task).  In  contrast,  a  trainee  with  a  performance  goal  orientation  is  more 
focused  on  demonstrating  high  ability  on  the  task  and  these  trainees  tend  to  be  more 
concerned  about  getting  a  positive  evaluation  and  outperforming  others  while  showing 
less  effort. 

Trainees  with  higher  LGO  have  been  shown  to  reach  greater  levels  of  learning, 
knowledge  development,  skill  acquisition,  and  performance  (e.g.,  Bell  &  Kozlowski, 
2002b).  A  number  of  research  findings  may  shed  light  on  this  relationship.  LGO  is 
positively  related  to  feedback-seeking  behavior  (VandeWalle  &  Cummings,  1997; 
VandeWalle,  Ganesan,  Challagalla,  &  Brown,  2000),  having  more  complex  learning 
strategies  (Ames,  1992;  Fisher  &  Ford,  1998;  Payne,  Youngcourt,  &  Beaubien,  2007), 
and  adaptive  performance  (Kozlowski  et  al.,  2001).  LGO  is  also  positively  related  to  self- 
efficacy  (Bell  &  Kozlowski,  2002b;  Kozlowski,  Gully,  et  al,  2001;  Phillips  &  Gully,  1997; 
Potosky  &  Ramakrishna,  2002),  enjoyment  of  the  activities  (Barron  &  Harackiewicz, 
2001),  and  motivation  to  learn  (Colquitt  &  Simmering,  1998).  Higher  LGO  trainees  tend 
to  react  more  positively  to  learning  opportunities,  choose  more  challenging  tasks, 
persist  in  the  tasks  they  choose  (Farr,  Hofmann,  &  Ringenbach,  1993;  Phillips  &  Gully, 

1997) ,  and  tend  to  remain  motivated  in  the  face  of  performance  difficulties  (Colquitt  & 
Simmering,  1998),  perhaps  because  they  use  self-regulation  strategies  to  maintain  their 
interest  and  motivation  for  the  learning  task  (Zimmerman  &  Martinez-Ponz,  1990). 
They  use  metacognitive  strategies  more  than  high  PGO  trainees  (Ford,  Smith, 
Weissbein,  Gully,  &  Salas,  1998)  and  tend  to  set  advanced  or  stretch  goals  (Brett  & 
VandeWalle,  1999;  Phillips  &  Gully,  1997).  LGO  is  not,  however,  always  beneficial;  Bell 
and  Kozlowski  (2002b)  found  that  trainees’  general  cognitive  ability  influenced  the 
effectiveness  of  LGO.  High  ability  trainees  can  benefit  from  LGO  and  tend  to  show 
increases  in  self-efficacy  and  in  learning  performance;  however,  lower  ability  trainees 
with  high  LGO  experienced  deficits  in  both  self-efficacy  and  learning  performance. 

Higher  PGO  trainees  tend  to  have  lower  motivation  to  learn  (Colquitt  &  Simmering, 

1998)  and  show  more  anxiety  during  training  (Chen,  Gully,  Whiteman,  &  Kilcullen, 
2000).  They  tend  to  choose  tasks  they  perceive  to  be  easier  and  more  likely  achievable 
(Phillips  &  Gully,  1997).  Some  research  shows  a  negative  relation  of  PGO  with  self- 
efficacy  (Payne  et  al.,  2007;  Phillips  &  Gully,  1997)  and  a  hesitance  to  seek  feedback, 
especially  when  higher  PGO  trainees  are  also  high  in  performance  avoidance  orientation 
(Payne  et  al.,  2007;  VandeWalle  &  Cummings,  1997).  Overall,  PGO  has  been  associated 
with  lower  performance,  relative  to  LGO  (c.f.,  Bell  &  Koslowski,  2002b).  One  reason 
could  be  that  it  has  been  shown  to  be  positively  related  to  task-related  cognitive 
interference  (Hofmann,  1993). 
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Initial  hypotheses  on  LGO  as  compared  to  PGO  were  that  trainees  could  not  be 
simultaneously  high  or  low  on  both  orientations  at  the  same  time,  thus  measures  for  the 
two  goal  orientations  tended  to  put  the  two  orientations  on  a  single  dimension.  When 
researchers  questioned  this  assumption,  they  found  that  the  choice  of  learning  goals  was 
not  mutually  exclusive  and  trainees  could  have  multiple  goals  (Button,  Mathieu,  & 

Zajac,  1996).  They  also  found  that  high  PGO  is  not  always  a  detriment.  PGO  is  positively 
associated  with  performance,  especially  when  LGO  is  also  high  (Elliot,  1997;  Elliot  & 
Church,  1997;  Elliot  &  Harackievicz,  1996).  For  example,  when  trainees  were  both  high 
in  LGO  and  low  in  PGO,  they  had  higher  levels  of  performance  after  practice;  however, 
when  they  were  high  in  both  LGO  and  PGO,  somewhat  paradoxically,  they  had  the 
lowest  levels  of  performance  after  practice  (Yeo  &  Neal,  2004). 

More  recently,  the  2-goal  model  has  been  extended  to  include  an  approach- 
avoidance  distinction.  Approach  goals  are  focused  on  positive  outcomes  and  avoidance 
goals  focused  on  avoiding  negative  outcomes.  At  first,  only  PGO  was  partitioned  into 
separate  approach  goals,  referred  to  as  a  prove  performance  orientation  (PPO)  in  which 
trainees  want  to  prove  themselves  by  performing  well  and  an  avoid  performance  goal 
orientation  (APGO)  with  trainees  seeking  to  avoid  mistakes  (Elliot  &  Church,  1997; 

Elliot  &  Harackiewicz,  1996;  VandeWalle,  1997).  In  this  3-goal  model,  LGO  was 
considered  to  be  only  an  approach  goal.  Research  has  shown  that  PPO  is  related  to 
achievement  in  learning  contexts  when  conceptualized  as  a  goal  to  perform  better  than 
others,  although  it  has  been  found  to  be  unrelated  to  some  adaptive  variables  such  as 
deep  processing  (Elliot,  McGregor  &  Gable,  1999;  Harackiewicz,  Barron,  &  Elliot,  1998; 
Harackiewicz  Barron,  Tauer,  Carter,  &  Elliot,  2000)  and  intrinsic  motivation  (Church, 
Elliot,  &  Gable,  2001;  Elliot  &  Church,  1997;  Harackiewicz,  Barron,  Carter,  Lehto,  & 
Elliot,  1997;  Tanaka  &  Yamauchi,  2001).  In  a  review  of  the  literature  supporting  the 
separation  of  PGO  into  PPO  and  APGO,  Harackiewicz  and  colleagues  found  that  LGO 
and  PPO  independently  promoted  different  learning  outcomes  and  found  that  optimal 
motivation  occurred  when  trainees  had  adopted  both  goal  orientations.  APGO  has, 
however,  been  found  to  be  associated  with  many  negative  learning  outcomes  that  PGO 
had  before:  higher  anxiety,  lower  learning  performance,  and  maladaptive  behaviors 
after  failure  such  as  learned  helplessness  and  self-handicapping  (Barron  & 

Harackiewicz,  2001;  Elliot  &  Church,  1997). 

Even  more  recently,  a  full  2x2  learning  goal  model  was  proposed  in  which  LGO  was 
divided  in  terms  of  approach  and  avoidance;  approach  LGO  (ApLGO)  was 
conceptualized  as  a  focus  on  task  mastery  with  avoidance  LGO  (AvLGO)  a  focus  on  not 
doing  worse  than  before  (Elliot  &  McGregor,  2001).  Performance-approach  goals 
correspond  to  positive  learning  strategies  and  learning  outcomes  and  are  unrelated  to 
negative  learning  processes  and  outcomes,  whereas  performance-avoidance  goals  are 
more  likely  to  be  linked  to  a  negative  achievement  pattern  (Elliot  &  McGregor,  1999; 
Elliot  &  Moller,  2003).  For  example,  ApLGO  has  been  positively  linked  to  the  tendency 
to  process  information  in  depth  and  to  the  more  frequent  and  effective  use  of 
metacognitive  strategies  and  negatively  to  procrastination.  In  contrast,  AvLGO  was 
positively  linked  to  the  tendency  to  process  information  shallowly,  and  negatively 
associated  with  the  use  of  metacognitive  strategies  and  procrastination  behaviors 
(Howell  &  Watson,  2007). 
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There  have  been  several  moderators  of  the  high  PGO-performance  relationship. 
Individual  differences  may  moderate  the  relationship  between  PGO  and  learning 
performance.  For  example,  self-efficacy  may  improve  the  learning  performance  of  high 
PGO  trainees;  trainees  with  a  high  PGO  who  also  have  high  self-efficacy  practiced  more 
than  trainees  with  a  high  PGO  and  low  self-efficacy  (Brown,  2001;  DeRouin  et  al., 

2005). 

The  complexity  of  the  task  can  be  important:  High  PGO  trainees  can  outperform 
high  LGO  trainees  when  the  learning  task  is  relatively  simple  (Seijts,  &  Latham,  2001; 
Seijts,  Latham,  Tasa,  &  Latham,  2004;  Winters  &  Latham,  1996),  however,  high  PGO 
trainees  did  not  perform  as  well  as  high  LGO  trainees  when  the  task  was  complex 
(Steele-Johnson,  Beauregard,  Hoover,  &  Schmidt,  2000).  Cognitive  resources  are 
generally  limited  and  when  the  task  is  difficult,  specific  performance  goals  should  not  be 
used  in  the  early  phases  of  learning  a  new  task  since  such  goals  were  found  to  require 
significant  amounts  of  cognitive  resources  in  complex  tasks  (Deshon  &  Alexander,  1996, 
Kanfer  &  Ackerman,  1989). 

The  fit  between  training  conditions  and  trainee  personalities  and  learning  goal 
orientations  can  impact  learning  performance;  individuals  high  in  PGO  increased  their 
learning  performance  when  the  specificity  of  feedback  was  increased  and  the  effects  of 
feedback  specificity  was  greatest  for  high-PGO  trainees  who  were  also  low  in  LGO 
(Davis,  Carson,  Ammeter,  &  Treadway,  2005).  Learning  goals  can  also  be  influenced  by 
instructions.  People  can  be  helped  to  discover  needed  strategies  by  assigning  them 
difficult  and  specific  learning  goals  instead  of  difficult  and  specific  outcome  goals  (Seijts 
&  Latham,  2001;  Seijts  et  al.,  2004;  Winters  &  Latham,  1996).  Self-esteem  is  often 
negatively  related  to  avoidance  goals  while  positively  related  to  approach  goals 
(Heimpel,  Elliot,  &  Wood,  2006). 

Need  for  Achievement 

Need  for  Achievement  includes  characteristics  like  perseverance  during  challenge, 
strong  work  behavior,  and  desire  for  excellence.  Mount  and  Barrick  (1995)  found  in  a 
meta-analysis  that  need  for  achievement  was  correlated  with  training  proficiency. 

Hough  (1998)  found  that  need  for  achievement  was  related  to  job  proficiency,  training 
success,  and  educational  success.  Need  for  achievement  has  also  been  linked  with 
training  motivation  (Carlson,  Bozeman,  Kacmar,  Wright,  &  McMahan,  2000).  Need  for 
achievement  is  related  to  the  constructs  of  ambition  and  achievement  motivation.  Not 
surprisingly,  ambition  has  been  shown  to  predict  academic  performance  (Driskell, 
Hogan,  Salas,  &  Hoskin,  1994).  Students  high  in  need  for  achievement  are  more  likely  to 
set  challenging  goals  for  their  class  performance  and  consequentially  are  more  likely  to 
obtain  better  grades  (Hollenbeck,  Williams,  &  Klein,  1989;  Phillips  &  Gully,  1997).  In 
Colquitt,  LePine,  and  Noe’s  (2000)  meta-analysis  concerning  training,  it  was  found  that 
achievement  motivation  was  moderately  linked  to  motivation  to  learn.  Trainees  high  in 
need  for  achievement  tended  to  endorse  learning  goals  and  were  less  likely  to  endorse 
performance  avoidance  learning  goals  (Payne  et  al,  2007). 
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Personal  Control  Beliefs 


Personal  control  beliefs  reflect  individuals’  beliefs  regarding  the  extent  to  which  they 
are  able  to  control  or  influence  events  to  receive  a  desired  outcome  (see  Skinner,  1996 
for  a  review).  They  are  central  concepts  in  several  theories  of  motivation  and  self¬ 
regulation,  including  expectancy  theory  (Vroom,  1964),  self-determination  theory  (Ryan 
&  Deci,  2000),  and  social-cognitive  theory  (Bandura,  2001).  Trainees  will  be  more 
motivated  to  perform  well  in  training  if  they  believe  that  their  efforts  will  lead  to  high 
quality  training  outcomes,  that  high  performance  in  training  will  also  lead  to  high  job 
performance,  and  that  high  job  performance  will  be  instrumental  in  obtaining  preferred 
outcomes  while  avoiding  unwanted  outcomes  (Noe,  1986).  Research  on  specific  control 
beliefs  is  summarized  below. 

General  Self-Efficacy  (GSE)  and  Self-Esteem 

Setting  realistic  goals  and  monitoring  progress  towards  these  goals  involves  self- 
efficacy,  a  belief  that  a  person  has  the  ability  to  organize  and  carry  out  the  actions 
required  to  achieve  one’s  goals  (Bandura,  1997).  Self-efficacy  can  be  conceptualized  as 
one’s  belief  about  performance  capabilities  in  a  particular  domain  or  on  a  general  belief 
of  their  performance  capabilities  across  tasks  (GSE).  Self-efficacy  beliefs  are  related  to 
motivation  to  learn,  learning  performance,  and  job  performance  (Colquitt  et  al.,  2000), 
as  well  as  phenomena  related  to  learning  such  as  self-regulatory  behavior  (Malpass, 
O'Neil  &  Hocevar,  1999)  and  transfer  of  training  (Gist  et  al.,  1991).  Self-esteem  also 
mediates  the  relationship  between  goal  orientation  and  performance;  trainees  high  in 
performance  goal  orientation  who  also  have  high  self-efficacy  beliefs,  practiced  the  most 
in  an  Internet  training  class  while  trainees  who  were  high  in  performance  orientation 
and  had  low  self-efficacy  beliefs,  practiced  the  least  (Brown,  2001).  Self-efficacy  has 
been  shown  to  influence  trainees’  decision  making  during  training.  For  instance,  higher 
self-efficacy  has  been  shown  to  be  related  to  the  more  frequent  and  effective  use  of 
learning  strategies  (Schunk  &  Ertmer,  2000;  Zimmerman,  Bandura,  &  Martinez-Pons, 
1992).  Training  interventions  directed  at  increasing  trainees’  task-specific  self-efficacy 
were  more  effective  among  trainees  with  low,  rather  than  high,  GSE  (Eden  &  Aviram, 
1993;  Eden  &  Zuk,  1995). 

Self-esteem  is  an  affective8  component  of  self-evaluation  and  is  a  belief  of  self-worth 
and  liking  or  disliking.  Research  by  Chen,  Gully,  and  Eden  (2004)  suggests  that  GSE 
may  be  more  strongly  related  to  motivational  mechanisms  such  as  task  specific  self- 
efficacy,  self-set  goals,  and  effort  allocation,  whereas  self-esteem  is  relatively  more 
strongly  related  to  emotional  mechanisms,  such  as  state  anxiety  and  emotional 
regulation. 


8  Affect  refers  to  the  experience  of  feelings  or  emotions.  The  affective  domain  represents  one  of  the  three 
divisions  described  in  modern  psychology:  the  cognitive,  the  conative,  and  the  affective.  The  cognitive 
domain  is  closely  related  to  abstract  concepts  such  as  mind,  reasoning,  perception,  intelligence,  learning, 
and  information  processing.  The  conative  domain  includes  processes  such  as  impulse,  desire,  volition, 
and  striving.  The  affective  domain  includes  motives,  attitudes,  moods,  and  emotions. 
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Locus  of  Control 


Locus  of  control  (LOC)  is  a  generalized  belief  about  one’s  personal  efficacy.  It  refers 
to  the  extent  to  which  individuals  believe  that  they  can  control  events  that  affect  them. 
Individuals  with  an  internal  LOC  tend  to  believe  that  performance  outcomes  are 
contingent  on  their  own  behaviors  and  individuals  with  external  LOC  tend  to  believe 
that  performance  outcomes  are  more  situational  and  beyond  their  control  (Rotter, 

1966).  Those  with  extreme  external  LOC  often  believe  that  powerful  others,  fate,  or 
chance  primarily  determine  events.  Trainees  with  greater  internal  LOC  have  more 
positive  attitudes  toward  training  (Noe  &  Schmitt,  1986)  and  have  higher  self-efficacy 
beliefs  (Phillips  &  Gully,  1997).  Internal  LOC  has  been  found  to  be  more  related  to  skill 
acquisition  than  to  external  LOC  (Silver,  Mitchell,  &  Gist,  1995).  Internal  LOC  was  more 
strongly  related  to  motivation  to  learn,  whereas  external  LOC  was  moderately  related  to 
declarative  knowledge  and  transfer  of  training  (Colquitt  et  al.,  2000).  However,  internal 
LOC  is  trainable;  for  instance,  computer-based  instruction  can  facilitate  a  perception  of 
internalization  of  control,  especially  with  disadvantaged  learners  (Swan,  Mitrani, 
Guerrero,  Cheung,  &  Schoener,  1990).  Of  special  interest  to  this  literature  review, 
research  has  shown  that  individuals  with  internal  LOC  tend  to  significantly  outperform 
external  LOC  individuals  in  computer-based  training  (Santiago  &  Okey,  1992;  Wang  & 
Newlin,  2000). 

I  nstru  mentality 

Instrumentality  is  the  belief  that  higher  training  performance  will  result  in 
rewarding  outcomes.  Unfortunately,  although  much  research  has  been  conducted  to 
confirm  the  influence  of  instrumentality  on  motivation  in  general,  little  research  has 
examined  the  effect  of  instrumentality  beliefs  on  training  outcomes.  In  one  study, 
Mathieu,  Tannenbaum,  and  Salas  (1992)  found  no  relationship  between  instrumentality 
beliefs  and  performance  in  a  training  class,  but  thought  that  the  topic  (how  to 
proofread)  may  not  have  been  perceived  as  instrumental  to  a  successful  career  in  clerical 
and  administrative  assistant  positions.  However,  a  second  study  by  Chiaburua  and 
Lindsay  (2008)  found  that  instrumentality  was  a  crucial  indicator  of  both  motivation  to 
learn  and  transfer  of  training.  The  conflicting  results  leave  the  value  of  instrumentality 
uncertain. 

Learning  Styles 

The  term  “learning  styles”  (also  known  as  cognitive  styles)  refers  to  the  concept  that 
people  differ  with  regard  to  what  mode  of  instruction  or  study  is  most  effective  for  them. 
Advocates  of  learning-style  assessment  contend  that  ideal  instruction  requires 
identifying  an  individual’s  learning  style  and  tailoring  instruction  to  the  learning  style. 
Assessments  of  learning  style  typically  ask  people  the  sort  of  information  presentation 
they  prefer  (e.g.,  words  versus  pictures)  and  what  type  of  mental  activity  they  find  most 
engaging.  The  learning-styles  perspective  has  acquired  great  sway  with  educators  and 
trainers  ranging  from  kindergarten  to  graduate  school  practitioners.  The  followers  of 
specific  learning  style  theories  (e.g.,  Kolb,  1984)  are  often  enthusiastic  devotees.  There  is 
a  thriving  industry  publishing  learning-styles  instruments  and  guidebooks,  as  well  as 
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providing  professional  development  workshops  on  specific  learning-styles  for  trainers, 
teachers,  and  educators. 

Currently,  research  supporting  learning  styles  is  controversial.  Coffield,  Moseley, 
Hall,  &  Ecclestone  (2004)  conducted  an  extensive  review  of  the  research  literature 
funded  by  the  Department  for  Education  and  Skills,  United  Kingdom.  In  the  review, 
they  identified  71  different  theories  of  learning  styles  and  closely  reviewed  the  13  most 
popular  theories,  concluding  that  none  had  been  adequately  validated  through 
independent  research  (either  failing  to  meet  standards  for  either  construct  validity  or 
predictive  validity  and,  not  infrequently,  both)  and  most  of  the  associated  measures  had 
at  least  one  psychometric  weaknesses  (internal  consistency  or  test-retest  reliability).  A 
more  recent  review  of  the  learning  styles  literature  was  conducted  by  Pashler,  McDaniel, 
Rohrer,  and  Bjork  (2009).  The  authors  were  charged  with  determining  whether 
learning-styles  instruments,  practices,  and  theories  were  supported  by  scientific 
evidence.  They  found  many  problems  with  the  research  designs  and  resulting  validation 
evidence  for  learning  style  theories.  They  found  most  of  the  evidence  they  collected 
showed  training  methods  were  equivalent  across  different  styles  and,  where  there  was 
evidence  to  support  learning  styles,  the  evidence  was  weak  and  often  conflicting. 
Therefore  they  concluded  that  it  was  not  meaningful  or  cost-effective  to  conduct 
assessments  or  to  develop  training  specific  to  different  learning-styles. 

Personality  Traits 

Big  Five  Personality  Traits 

The  Big  Five  model  is  a  comprehensive,  data-driven  research  model  of  personality. 
The  Big-Five  model  (also  referred  to  as  the  Five  Factor  Model,  or  FFM)  was  created  by 
factor  analyzing  numerous  personality  trait  measures;  this  model  was  discovered 
independently  by  several  researchers  looking  for  a  taxonomy  of  personality  traits  (e.g., 
Digman,  1990;  Goldberg,  1990).  The  model  is  comprised  of  five  higher-order  traits  that 
are  each  composed  by  several  lower-level,  narrower,  and  more  specific  facets.  The 
predictability  of  the  five  personality  traits  have  been  found  to  be  both  broad  and 
comprehensive,  and  as  would  be  expected,  some  of  the  lower-level  facets  are  more 
predictive  than  the  broader  level  traits  in  some  situations. 

The  five  factors  are  extraversion,  agreeableness,  conscientiousness,  neuroticism,  and 
openness  to  experience  (also  referred  to  as  intellectance;  e.g.,  McCrae  &  Costa,  1992). 
People  who  score  high  on  extraversion  tend  to  have  positive  emotions,  be  assertive,  and 
have  the  tendency  to  seek  out  stimulation  and  the  company  of  others.  People  who  are 
agreeable  tend  to  get  along  with  other  people.  They  tend  to  feel  compassionate  and 
cooperative  towards  others  instead  of  suspicious  or  antagonistic.  Conscientious  people 
are  self-disciplined,  follow  rules,  and  they  tend  to  be  goal-driven.  Neuroticism  refers  to 
the  frequency  to  which  people  experience  negative  emotions.  People  who  are  neurotic 
are  emotionally  unstable,  with  their  feelings  changing  more  frequently  compared  to 
non-neurotic  individuals.  People  who  are  open  to  experience  are  imaginative,  curious, 
enjoy  culture  and  abstract  ideas,  and  are  more  sensitive  to  beauty.  They  tend  to  be  more 
aware  of  their  feelings  and  hold  more  unconventional  beliefs.  They  also  tend  to  seek  out 
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and  enjoy  new  ideas  and  situations  and  are  more  likely  to  change  their  beliefs  after 
doing  so. 

In  Barrick  and  Mount’s  (1991)  meta-analysis,  extroversion,  conscientiousness,  and 
openness  to  experience  were  all  positively  related  to  better  training  performance  while 
emotional  stability  and  agreeableness  all  had  smaller  results.  However,  another  meta¬ 
analysis  showed  that  while  conscientiousness  was  positively  related  to  motivation  to 
learn  it  was  negatively  related  to  actual  training  performance  (Colquitt  et  al.,  2000). 
Several  possible  explanations  exist  for  the  mixed  results  for  conscientiousness.  While 
several  of  the  facets  of  the  conscientiousness  trait  generally  reflect  behavioral  tendencies 
characteristic  of  successful  self-regulation  (Roberts,  Chernyshenko,  Stark,  &  Goldberg, 
2005),  some  facets,  (e.g.,  dependability,  order,  and  dutifulness)  have  been  shown  to 
reduce  performance  when  adaptability  to  changing  task  conditions  is  important 
(LePine,  Colquitt,  &  Erez,  2000).  Martocchio  and  Judge  (1997)  found  that 
conscientiousness  was  positively  related  to  self-deception  in  training  settings,  with 
trainees  higher  in  conscientiousness  tending  to  overestimate  their  level  of  learning, 
which  was  then  negatively  related  to  learning  outcomes.  Conscientiousness  can  mean 
that  fewer  tasks  are  getting  done,  since  conscientious  trainees  take  longer  to  complete  a 
given  set  of  tasks  (Driskell  et  al.,  1994).  Conscientiousness  also  interacts  with  previous 
performance  and  with  emotional  stability  to  explain  performance;  when  early 
performance  is  weak,  higher  conscientiousness  has  the  greatest  effect  on  learning 
performance,  compared  to  trainees  with  higher  conscientiousness  that  performed  well 
early  on  (Herold,  Davis,  Fedor,  &  Parsons,  2002). 

Higher  anxiety  levels  have  been  associated  with  lower  learning  performance 
outcomes  (Chen  et  al.,  2000;  Martocchio,  1994;  Colquitt  et  al,  2000),  and  trainees  who 
have  higher  state  anxiety  or  who  have  higher  emotional  instability  (i.e.,  neuroticism) 
show  performance  deficits  in  training.  However,  context  can  negate  the  performance 
deficit  shown  by  anxious  students;  they  perform  better  when  given  more  structured 
learning  environments  but  perform  less  effectively  with  relatively  unstructured  training 
(Snow,  1989, 1991).  Kanfer  and  associates  theorize  that  anxiety  affects  performance  by 
diverting  attentional  focus,  thus  reducing  resources  available  for  relevant  “on-task” 
learning  (Kanfer  &  Ackerman,  1991;  Kanfer  &  Heggestad,  1997).  However,  there  is 
evidence  that  showing  trainees  how  to  handle  errors  does  lead  to  better  emotional 
control  and  higher  learning  performance.  When  error  management  training  is  provided, 
it  can  lower  state  anxiety  and  improve  self-efficacy  as  well  as  performance  (Bell  & 
Kozlowski,  2008).  Trainees  who  have  instruction  on  howto  manage  learning  errors 
show  evidence  of  having  acquired  greater  emotional  control  and  metacognitive  skills 
and  will  then  outperform  those  who  have  not  had  error  management  training  (Keith  & 
Frese,  2005).  Neuroticism  was  shown  to  be  positively  related  to  avoidance  goals  in 
general  and  negatively  associated  with  approach  goals,  while  extraversion  was  related  to 
neither  type  of  goal  (Heimpel  et  al.,  2006). 
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Emotional  I  ntelligence 

Goleman  (1995)  defined  emotional  intelligences  (El)  as  “the  capacity  for  recognizing 
our  own  feelings  and  those  of  others,  for  motivating  ourselves,  and  for  managing 
emotions  well  in  ourselves  and  in  our  relationship.”  He  described  five  dimensions:  three 
are  personal  competencies  (self-awareness,  self-regulation,  and  a  motivational 
competency)  and  two  are  social  competencies  (empathy  and  social  skills)  (Goleman, 
1998).  Although  there  is  controversy  over  El  (c.f.,  Cote  &  Miners,  2006;  Fox  &  Spector, 
2000;  Locke,  2005;  Schulte,  Ree,  &  Carretta,  2004),  there  is  also  evidence  that  El  is 
both  distinct  from  other  ability  and  personality  measures  and  related  to  learning 
performance  (Austin,  Evans,  Goldwater,  &  Potter,  2005;  Law,  Wong,  &  Song,  2004; 
Petrides,  Fredrickson,  &  Furnham,  2004).  For  instance,  for  first  year  medical  students, 
El  was  positively  related  to  attitudes  about  training  and  on  exam  performance  early  in 
the  year,  but  not  later  in  the  year  (Petrides  et  al.,  2004).  Nevertheless,  there  are  many 
controversies  concerning  the  structure,  measurement,  and  stability  of  El  so  that  it  has 
not  been  broadly  studied  in  educational  settings. 

Tolerance  for  Ambiguity 

Tolerance  for  ambiguity  is  related  to  how  individuals  react  to  uncertain  situations  or 
stimuli.  The  person  with  low  ambiguity  tolerance  tends  to  perceive  ambiguous  stimuli  as 
threatening  and  stressful,  and  tends  to  react  prematurely  to  avoid  ambiguity,  while  a 
person  with  high  tolerance  for  ambiguity  perceives  the  same  stimuli  as  interesting  and 
challenging  (Furnham  &  Ribchester,  1995).  When  trainees  encounter  new  concepts  and 
ideas,  they  often  do  not  understand  all  the  new  information,  concepts  sometimes  appear 
to  have  multiple  meanings  and  are  vague  (Chapelle  &  Roberts,  1986;  Grace,  1998). 

There  is  also  a  tendency  for  those  low  in  tolerance  of  ambiguity  to  perceive  ambiguous 
situations  rigidly  in  “black  or  white”  terms  (Bhushan  &  Amal,  1986).  Students  who  are 
tolerant  of  ambiguity  are  more  willing  to  take  risks  (Naiman,  Frohlich,  Stern,  &  Todesco, 
1978;  Rubin,  1975;  Stern,  1975;)  and  show  endurance  on  tasks,  and  have  higher  levels  of 
achievement  (Chapelle  &  Roberts,  1986;  Naiman,  Froclich,  &Todeso,  1975).  Tolerance 
for  ambiguity  is  also  positively  related  to  the  academic  success  of  undergraduates  (Boyd, 
Hunt,  Kandell,  &  Lucas,  2003).  Students  who  have  a  tolerance  for  ambiguity  are  more 
likely  to  be  high  in  LGO  and  lower  in  PGO  (Kroll,  1988). 


9  Goleman  receives  the  popular  credit  for  the  term  “emotional  intelligence”  because  of  his  bestselling 
book.  However,  in  the  1920s  E.  L.  Thorndike  used  the  term  “social  intelligence”  in  a  similar  way.  In  1983, 
Howard  Gardner's  Frames  of  Mind:  The  Theory  of  Multiple  Intelligences  introduced  Interpersonal 
Intelligence  (understanding  the  motivations  of  other  people)  and  Intrapersonal  Intelligence 
(understanding  your  own  motivations).  Stanley  Greenspan  also  used  the  term  “emotional  intelligence”  in 
1989  in  his  book  The  Development  of  the  Ego:  Implications  for  Personality  Theory,  Psychopathology, 
and  the  Psychotherapeutic  Process. 
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Adaptability/  Flexibility 


Individuals  who  are  adaptable  or  flexible  are  more  willing  to  change  approaches  to 
tasks,  enjoy  variety,  and  are  able  to  handle  multiple  demands.  Adaptability  is  related  to 
overall  training  outcomes  (White  et  al.,  2002).  Carlson,  Bozeman,  Kacmar,  Wright,  and 
McMahan  (2000)  found  adaptability  was  positively  correlated  with  motivation  to  learn 
and  reactions  to  training  (see  also,  Stone,  Kemmerer,  &  Guetal,  1984).  While,  in  the 
literature,  the  definition  of  the  construct  varies  somewhat,  Houston,  Borman,  Farmer, 
and  Bearden  (2006)  theorized  that  adaptability  was  essential  to  success  in  the  Navy 
because  of  the  constantly  changing  nature  of  jobs  and  evolving  technologies  used  on 
those  jobs.  For  example,  Herold,  Davis,  Fedor,  and  Parsons  (2002)  highlighted  that 
training  programs,  like  the  pilot  training  program  they  studied,  are  multifaceted,  with 
different  stages  and  different  environments  (e.g.,  classroom,  simulator,  aircraft  with 
instructor).  At  each  level,  the  individual  characteristics,  both  cognitive  and  non- 
cognitive,  interact  and  produce  different  results.  However,  adaptability  was  not  related 
to  academic  performance  at  different  time  points,  but  was  negatively  related  to  rate  of 
academic  growth  over  time.  Low  adaptability  students  had  more  difficulty  at  the 
beginning  of  their  education  when  there  is  the  greatest  amount  of  change  and  novel 
experiences,  but  they  improved  later  in  training.  In  contrast,  high  adaptability  students 
adapted  more  quickly  early  in  training,  but  had  little  room  for  improvement  later  in 
training  (Shivpuri,  Schmitt,  Oswald,  &  Kim,  2006).  While  the  low  adaptive  students 
were  able  to  struggle  through  pilot  training,  there  are  few  training  pipelines  that  are  two 
or  more  years  in  length.  For  a  more  typical  (enlisted)  training  pipeline  of  7-8  weeks, 
adaptability  would  be  even  more  important  since  there  is  little  time  to  recover  from 
early  difficulties. 

Perceptiveness/  Depth  of  Thought 

Perceptiveness/Depth  of  Thought  is  defined  as  being  interested  in  abstract  thinking, 
in  knowledge  outside  of  one’s  specialty,  and  in  understanding  how  things  work. 
Perceptiveness/Depth  of  Thought  has  been  correlated  with  educational  success  (Hough, 
1992, 1998).  The  construct  is  related  to  openness  to  experience  or  intellectance. 

Cognitive  Playfulness 

Cognitive  playfulness  is  the  tendency  to  “play”  with  new  information  or  a  problem  by 
testing  ideas  and  reviewing  hypotheses  (Csikszentmihalyi,  1990)  and  has  been  used  in 
research  predominantly  in  the  context  of  human-computer  interactions.  Martocchio  and 
Webster  (1992)  have  found  that  cognitive  playfulness  is  associated  with  learning 
performance  and  positive  affect  in  CBT  and  that  trainees  low  in  playfulness  benefit  more 
from  positive  feedback  compared  to  trainees  high  in  cognitive  playfulness  (Martocchio  & 
Webster,  1992;  Webster  &  Martocchio,  1992).  While  interesting,  this  construct  is  not 
well  researched  and  statistical  relationships  with  a  broad  range  of  training  or 
educational  outcomes  have  not  been  established. 
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Emotional  Control 


Emotional  control  (also  known  as  self-control)  is  the  ability  to  suppress  negative 
emotions  and  inappropriate  behaviors,  even  in  situations  where  it  is  difficult  to  do  so, 
and  to  think  before  acting.  Emotional  control  was  found  to  have  a  small  nonlinear 
relationship  to  academic  performance  at  2-  and  4-year  institutions  over-  and  under¬ 
inhibition  of  strong  feelings  being  detrimental  to  a  student’s  ability  to  perform 
academically  (Robbins,  Allen,  Casillas,  Peterson,  &  Le,  2006).  They  also  found  that  the 
relationship  between  emotional  control  and  retention  was  stronger  and  it  was  found  to 
be  a  stronger  predictor  than  academic  self-confidence. 

Emotional  control  can  be  trained.  Trainees  who  received  instructions  to  use  an 
emotion-control  strategy  designed  to  increase  the  frequency  of  positive  thoughts  and 
reduce  the  frequency  of  negative  thoughts  had  better  training  performance  (Kanfer  & 
Ackerman,  1989).  Error  management  instructions  can  improve  trainees’  ability  to 
handle  negative  emotions  (Bell  &  Kozlowski,  2008;  Keith  &  Frese,  2005),  which  then 
improves  transfer  of  training  (Keith  &  Frese,  2005). 

Positive  and  Negative  Affect 

Affective  disposition  is  the  tendency  for  an  individual  to  have  emotional  responses 
that  tend  to  follow  a  pattern  across  time  and  situations.  Measured  as  an  individual  trait, 
positive  affect  denotes  an  individual  with  high  energy  who  tends  to  experience  positive 
moods,  while  negative  affectivity  is  characterized  by  an  individual  who  tends  to  be  more 
in  distress  and  experience  negative  moods  (Watson,  Clark,  &Tellegen,  1988).  People 
who  tend  to  experience  positive  affect  tend  to  have  judgments  that  are  more  positive  and 
give  feedback  that  is  more  favorable  because  they  interpret  situations  more  positively 
and  more  optimistically  than  people  who  tend  not  to  experience  positive  affect  (Isen  & 
Patrick,  1983;  Petty,  Schumann,  Richman,  &  Strathman,  1993;  Weiss,  Nicholas,  &  Daus, 
1999),  suggesting  that  positive  affect  may  be  related  to  reactions  to  training.  Positive 
affect  has  a  facilitative  effect  on  intrinsic  motivation  by  influencing  the  cognitive  process 
involved  in  motivation,  namely  the  evaluation  of  the  rewards  and  beliefs  about  self- 
efficacy  and  instrumentality  (Erez  &  Isen,  2002;  Isen  &  Reeve,  2005).  Therefore, 
positive  affect  may  increase  intrinsic  motivation  during  training.  While  positive  and 
negative  affect  state  measures  have  been  used  in  training  studies  (e.g.,  Bretz  & 
Thompsett,  1992),  to  date  trait  measures  have  not. 

Creativity 

Most  research  looking  at  training  and  creativity  focuses  on  the  trainability  of 
creativity  (e.  g.,  Scott,  Leritz,  &  Mumford,  2004)  or  how  training  contributes  to 
creativity  (e.  g.,  Weisberg,  1999).  There  have  been  a  few  early  studies  that  suggest 
creative  thinking  contributes  to  academic  achievement  (Feldhusen,  Treffinger,  van 
Mondfrans,  &  Ferris,  1971;  Yamamoto,  1964),  but  that  line  of  research  ended  and  no 
study  to  date  has  examined  if  creativity  would  improve  training  outcomes  in 
organizations.  LGO  has  recently  been  associated  with  creative  performance  (Gong, 
Huang,  &  Farh,  2009;  Hirsch,  G.,  van  Knippenberg,  &  Zhou,  2009).  Generally  speaking, 
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there  is  no  agreement  on  how  to  measure  creativity  broadly  which  has  limited  the 
amount  and  consistency  of  research. 

J  ob- Knowledge  and  Work  Experience 

Previous  knowledge  of  a  topic  has  a  strong  positive  effect  on  training  performance, 
predicting  both  the  rate  of  learning  and  the  amount  learned  (Schmidt,  Hunter,  & 
Outerbridge,  1986).  Trainees  with  previous  knowledge  tend  to  use  advanced  strategies 
during  learning,  while  trainees  with  less  knowledge  tend  to  approach  learning  by 
initially  developing  a  conceptual  framework  of  the  domain  (Alexander,  Jetton,  & 
Kulikowich,  1995;  Murphy  &  Alexander,  2002).  Prior  domain  knowledge  was  positively 
related  to  monitoring  and  planning  activities  and  negatively  related  to  the  use  of 
strategies  during  a  hypermedia  CBT  activity  (Moos  &  Azevedo,  2008).  Trainees  with 
lower  prior  domain  knowledge,  in  general,  tended  to  use  less  varied  strategies  during 
learning  (Alexander  et  al.,  1995;  Murphy  &  Alexander,  2002).  In  essence,  prior 
experience  allows  the  early  adoption  of  a  good  strategy  (with  self-monitoring  to  ensure  it 
is  working)  whereas  a  lack  of  prior  experience  leads  to  strategy  switching  but  the 
changes  are  among  a  small  set  of  “logical”  approaches.  At  least  part  of  the  difficulty 
some  trainees  have  in  navigating  in  hypermedia  training  activities  has  been  found  to  be 
related  to  a  limited  understanding  of  the  conceptual  structure  of  the  domain  that  would 
otherwise  help  to  guide  their  interaction  with  hypermedia  (Chen,  Fan,  &  Macredie, 

2006;  Shapiro,  2004).  Previous  experience  with  information  helps  trainees  be  more 
efficient  in  their  use  of  training  materials  (Shih,  Munoz,  &  Sanchez,  2006)  but  its’  effect 
fades  by  the  end  of  training  (Ree,  Carretta,  &  Teachout,  1995;  Shih,  Munoz,  &  Sanchez, 
2006). 

Aptitudes 

The  benefits  of  including  intellectual  measures  in  decisions  about  selection  and 
training  assignments  are  undeniable  (Gottfredson,  1986).  The  benefits  for  the 
organization  include  reduced  training  failures  and  training  costs  and  greater 
performance  and  productivity  (Hunter,  1983).  For  the  individual,  the  benefits  include  a 
greater  probability  of  success  (in  training  and  on-the-job)  and  a  better  match  between 
their  abilities  and  their  training  and  job  assignments  (Ones  &  Viswesvaran,  2002).  The 
mechanism  for  the  success  of  assignments  based  on  intellectual  ability  is  well 
understood;  the  training  and  job  characteristics  tap  many  of  the  same  intellectual 
abilities  used  to  perform  the  tasks  (Hunter  &  Hunter,  1984). 

In  enlisted  military  selection  and  classification,  the  Armed  Services  Vocational 
Aptitude  Battery  (ASVAB)  provides  the  intellectual  assessment.  The  ASVAB  is  a  9-test 
multiple-aptitude  battery  that  has  been  used  to  predict  success  in  training  in  the  Navy 
(Driskell  et  al.,  1994;  Wolfe,  Moreno,  &  Segall,  2004),  the  Air  Force  (Besetsny,  Earles,  & 
Ree,  1993;  Earles  &  Ree,  1992),  the  Army  (Campbell,  1990),  and  for  civilian  (and 
military)  air  traffic  controllers  (Carretta  &  King,  2008).  Ackerman  and  associates  have 
found  that  the  ASVAB  tests  can  predict  learning  performance  (Ackerman,  2003).  Job- 
specific  aptitudes  such  as  table  reading,  technical  reading,  industrial  math,  and 
following  directions  (all  prior  ASVAB  tests)  were  associated  with  task  performance  of 
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apprentices  on  manufacturing  tasks  (Hattrup  &  Schmidt,  1990).  However,  since  ASVAB 
scores  are  used  to  make  assignments  of  recruited  Sailors  to  ratings,  and  hence  to 
training,  range-restriction  may  appear  to  limit  the  ability  of  ASVAB  tests  to  predict 
training  success  after  assignment  has  been  made,  although  statistical  corrections  can 
ameliorate  this.  In  the  Navy,  the  validity  of  the  ASVAB  to  predict  training  outcomes 
varies  from  about  0.30  (special  operations)  to  as  high  as  0.85  (Nuclear  Field).  While 
these  training  outcome  validities  are  very  good  for  any  intellectual  ability  measure,  there 
is  substantial  evidence  that  the  predictions  can  be  improved  by  including  non-cognitive 
measures  (e.g.,  Campbell,  1990). 

Work- related  Attitudes 

An  attitude  is  a  long-lasting  evaluation  of  a  specific  person,  a  group  of  people,  or 
objects.  Attitudes  have  an  emotional  aspect  (degree  of  like  or  dislike  of  the  object  in 
question),  a  behavioral  aspect  (how  you  behave  toward  the  object),  and  a  cognitive 
aspect  (beliefs  about  the  object),  and  most  attitudes  are  learned  responses  (Fazio,  1986). 

Organizational  Commitment 

Mowday,  Porter,  and  Steers  (1982)  defined  organizational  commitment  as  the 
“relative  strength  of  an  individual's  identification  with  and  involvement  in  a  particular 
organization”  (Mowday,  Porter,  &  Steers,  1982,  p.  27;  for  more  contemporary  views,  see 
Meyer  &  Allen,  1991, 1997).  The  more  committed  trainees  are  to  the  organization,  the 
more  likely  they  are  to  view  training  as  useful  for  themselves  and  the  organization 
(Colquitt  et  al.,  2000).  Organizational  commitment  is  positively  related  to  motivation  to 
learn,  transfer  of  learning,  and  to  reactions  to  training  (Facteau,  Dobbins,  Russell,  Ladd, 
&  Kudisch,  1995;  Tannenbaum  et  al.,  1991),  although  at  least  one  study  did  not  find  such 
a  relationship  (Carlson  et  al,  2000).  Gade  (2003)  provides  extensive  evidence  on  the 
importance  of  organizational  commitment  in  the  military  in  terms  of  retention, 
attrition,  training  and  job  performance. 

Organizational  J  ustice 

Organizational  justice  research  explores  how  perceptions  of  fairness  are  related  to 
organizational  outcomes  (Greenberg,  1990).  Perceptions  of  fairness  strongly  affect  the 
employees’  attitudes,  including  job  satisfaction,  turnover  intentions,  organizational 
commitment,  and  workplace  behavior  such  as  job  satisfaction,  evaluation  of  authority, 
citizenship  behavior,  withdrawal  cognitions,  and  job  performance  (Colquitt,  Conlon, 
Wesson,  Porter,  &  Ng,  2001).  Research  has  found  four  distinct  types  of  fairness 
perceptions:  distributive,  procedural,  interpersonal,  and  informational  justice  (Colquitt, 
2001).  Distributive  justice,  the  fairness  of  the  rewards  and  outcomes  received,  is 
generally  perceived  if  rewards  received  are  proportional  to  perceived  input  (Adams  & 
Jex,  1999).  Procedural  justice  refers  to  the  fairness  of  the  procedures  used  in  making 
decisions  about  rewards  (Leventhal,  1980).  The  interpersonal  treatment  people  receive 
as  procedures  are  followed  is  interactional  justice  (Bies  &  Moag,  1986),  while  the 
fairness  and  adequacy  of  explanations  of  the  procedures  and  decisions  is  called 
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informational  justice  (Greenberg,  1993).  Only  two  studies  have  linked  fairness 
perceptions  to  training  outcomes.  Quinones  (1995)  found  that  a  combined  measure  of 
distributional  and  procedural  fairness  of  training  assignment  was  positively  related  to 
motivation  to  learn.  Liao  and  Tai  (2006)  found  that  interactional  justice  mediated  the 
relationships  between  distributive  and  procedural  justice  and  motivation  to  learn. 

Attitudes  Towards  Training 

Similar  in  concept  to  affective  and  cognitive  reaction  measures  of  training,  attitudes 
towards  training  are  beliefs  held  towards  the  training  program  but  measured  before 
training  begins.  The  reputation  of  the  quality  of  the  training  program  has  been  linked  to 
training  motivation  (Facteau  et  al.,  1995).  Having  a  positive  attitude  towards  formal 
training  in  general  has  also  been  associated  with  increased  training  motivation  (Carlson 
et  al,  2000).  Self-efficacy  and  motivation  to  learn  is  affected  by  trainees’  expectations 
about  the  training  program  (Tannenbaum  et  al.,  1991),  suggesting  that  trainees’ 
expectations  about  training  developed  before  training  takes  place  should  be  considered. 


Discussion 


Based  upon  this  review  of  the  literature,  the  case  can  be  made  to  include  the 
following  measures  of  individual  differences: 

•  Motivation  to  learn,  preferably  before  actual  training 

•  Intrinsic  motivation 

•  Metacognitive  abilities,  including  self-reflection 

•  Goal  orientation,  specifically  measures  from  the  goal  orientation  model 

•  Control  beliefs  (self-efficacy,  locus  of  control) 

•  Personality  (Big  Five  and  lower-level  facets  of  conscientiousness;  flexibility, 
self-control,  emotional  control) 

•  Organizational  commitment  and  perceptions  of  fairness 

•  Attitudes  towards  training  (met  expectations) 

There  are  many  published  and  commercial  tests  available  that  could  be  used  to 
assess  these  constructs.  Unfortunately,  test  administration  and  scoring  could  be  very 
costly  and  since  many  of  the  measures  are  proprietary,  the  cost  for  using  them  would  be 
in  perpetuity  if  the  Navy  continued  their  use.  However,  many  of  these  measures  can  be 
found  in  a  battery  of  individual  difference  measures  developed  by  Navy  Personnel 
Research,  Studies,  and  Technology  (NPRST)  and  funded  by  the  Office  of  Naval  Research 
(ONR).  The  battery  of  19  traits  is  called  the  Navy  Computer  Adaptive  Personality  Scales 
(NCAPS).  This  battery  is  owned  by  Navy  and  would  entail  no  use  or  scoring  costs. 
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NCAPS  was  created  to  be  complementary  to  the  Armed  Forces  Vocational  Aptitude 
Battery  (ASVAB)  used  to  assign  all  military  recruits  to  enlisted  jobs  based  on  technical 
knowledge  and  intellectual  ability.  NCAPS  was  designed  to  predict  a  wide-range  of 
training  and  job  performance  outcomes  as  well  as  important  aspects  of  organizational 
behavior  (e.g.,  commitment,  integrity,  leadership).  Based  on  an  extensive  review  of  the 
research  literature,  functional  knowledge  of  the  Navy,  and  subject  matter  expert 
judgments  (Borman,  Hedge,  Ferstl,  &  Kaufman,  2002;  Borman,  Hedge,  Ferstl, 

Kaufman,  Farmer,  &  Bearden,  2003;  Ferstl  et  al.,  2003),  NCAPS  is  composed  of  traits 
relevant  to  performance  in  each  Navy  enlisted  rating  and  most  officer  designators  as 
well.  NCAPS  has  been  further  refined  based  on  psychometric  testing,  expert  evaluations, 
and  extensive  field  studies.10  Below  is  the  list  of  NCAPS  trait  measures;  descriptions  of 
the  attributes  of  high  and  low  scores  for  each  scale  are  supplied  in  Appendix  A. 

•  Achievement  (AV) 

•  Adaptability/Flexibility  (ADF) 

•  Attention  to  Detail  (ADL) 

•  Dependability  (DEP) 

•  Dutifulness/Integrity  (DUT) 

•  Self-Reliance  (SRL) 

•  Social  Orientation  (SO) 

•  Stress  Tolerance  (ST) 

•  Vigilance  (VIG) 

•  Willingness  to  Learn  (WTL) 

•  Leadership  Orientation  (LO) 

•  Self-Control  (SC) 

•  Perceptiveness/Depth  of  Thought  (PDT) 

•  Innovation  (INV) 

•  Initiative  (INI) 

•  Empathy  (EMP) 

•  Commitment  (COM) 

•  Positive  Self-Concept  (PSC) 

•  Tolerance  for  Ambiguity  (TA) 


10  Note  that  NCAPS  concept  and  design  won  the  prestigious  M.  Scott  Myers  Award  from  the  Society  for 
Industrial  and  Organizational  Psychology  for  “Innovative  talent  measurement  and  recognized  for 
contributions  to  applied  research  in  the  workplace”  (January  14,  2010,  Atlanta,  GA). 
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NCAPS  has  been  shown  to  be  psychometrically  sound  (Schneider  et  al.,  2006). 
Studies  have  found  that  the  NCAPS  traits  correlate  significantly  with  similar  traits  in 
commercial  and  published  measures.  Initial  validation  studies  have  demonstrated 
NCAPS’  ability  to  predict  learning  performance  in  university  classes  (Underhill,  2006) 
and  Navy  training  at  multiple  Learning  Centers  (Houston,  Borman,  Farmer,  &  Bearden, 
2006)  with  initial  results  being  very  promising  especially  for  ratings  with  high  attrition 
rates  such  as  Air  Traffic  Controllers,  Explosive  Ordinance  Disposal-Diver,  and  Basic 
Underwater  Demolition/SEAL  (BUD/S)  training  (Houston  et  al.,  2006;  Rice  &  Mottern, 
2008).  A  fleet  pilot  study  demonstrated  that  NCAPS  traits  correlated  with  supervisors’ 
performance  ratings  (using  behaviorally  anchored  rating  scales)  across  a  variety  of  Navy 
enlisted  ratings  (Schneider  et  al.,  2006). 

NCAPS  has  the  added  benefit  of  being  web-enabled,  uses  items  presented  in  a 
paired-comparison  format,  which  is  scored  using  Item  Response  Theory  (IRT),  and  is 
administered  as  a  computer  adaptive  instrument.  NCAPS  uses  a  paired-comparison 
item  presentation  format  that  is  considered  more  difficult  to  “fake  good”11  than  standard 
Likert12  scales.  The  test  is  scored  using  a  modified  Zinnes  and  Griggs  ideal  point  paired- 
comparison  IRT  model  (Stark,  Chernyshenko,  &  Drasgow,  2006;  Stark  &  Drasgow, 
2002;  Underhill,  2006)  which  dramatically  reduces  the  number  of  items  necessary  to 
reliably  measure  a  trait.  In  addition,  the  test  is  adaptive  in  that  items  are  statistically 
selected  based  on  the  answer  to  previous  questions.  Finally,  the  19  traits  are  not 
measured  in  cohesive  blocks  of  items  (e.g.,  all  the  integrity  items,  followed  by  all  the 
self-control  items).  Rather,  trait  items  are  randomly  intermixed  which  further  increases 
the  difficulty  of  faking.  Several  experiments  have  been  conducted  that  demonstrate 
NCAPS  to  be  resistant  to  faking  and  socially  desirable  responding  (Underhill,  Bearden, 

&  Chen,  2008). 

The  combination  of  NCAPS  coverage  of  critical  individual  difference  constructs 
found  in  the  review,  Navy  ownership  of  the  test,  demonstrated  value  for  predicting  Navy 
relevant  outcomes  including  training,  careful  design  characteristics,  and  contemporary 
administration  all  argue  strongly  for  including  NCAPS  in  evaluations  of  computer-based 
training  and  job  performance  outcomes.  NCAPS  will  need  to  be  augmented  with  specific 
training  measures  to  obtain  the  full  spectrum  of  important  factors  that  affect  these 
outcomes. 


11  This  is  the  term  used  when  describing  people  trying  to  answer  questions  (fake)  to  impress  others, 
particularly  when  applying  for  jobs. 

12  Likert  scales  are  traditional  unidimensional  point  scales  varying  across  a  continuum,  e.g.,  from 
“strongly  disagree”  to  neutral  to  “strongly  agree”  where  the  respondent’s  task  is  to  rate  (usually 
themselves)  about  a  question  (e.g.,  I  work  harder  than  my  coworkers). 
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Appendix  A: 

NCAPS  Scales  and  Descriptions  of  the  Attributes  of  High 

and  Low  Scores 


A-o 


Table  A- 1 

NCAPS  scales  and  descriptions  of  the  attributes  of  high  and  low  scores 


Scales 

High  Scores 

Low  Scores 

Achievement  (AV) 

Like  to  set  challenging  goals; 
work  hard  over  long  periods  of 
time  when  necessary  to  achieve 
goals;  persists  in  the  face  of 
significant  obstacles  that  would 
cause  others  to  give  up;  strives 
for  excellence;  confident  in  ability 
to  perform  well. 

Avoid  challenging  goals  and 
projects;  prefer  to  work  only  as 
hard  as  necessary  to  complete 
projects  and  tasks;  gives  up  easily 
when  confronted  with  obstacles; 
doubt  their  ability  to  perform  well; 
display  little  ambition. 

Adaptability  and 
Flexibility  (ADF) 

Are  willing  to  change  their 
approach  to  tasks  and  projects; 
like  considerable  variety  at  work; 
adapt  readily  to  changes  in  their 
environment  involving  additional 
constraints,  multiple  demands, 
and  unanticipated  adversity. 

Like  to  do  things  the  way  they 
always  do  them;  have  difficulty 
adjusting  to  new  people, 
situations,  and  environments;  do 
not  adapt  well  to  changes  in  their 
environment  involving  additional 
constraints,  multiple  demands,  or 
unanticipated  adversity. 

Attention  to 

Detail  (ADL) 

Are  exacting,  precise,  and 
accurate;  spot  minor 
imperfections  or  errors;  are 
meticulous  and  thorough  in  their 
approach  to  tasks. 

Are  sloppy  and  imprecise;  miss 
important  details;  make  careless 
errors. 

Dependability 

(DEP) 

Are  reliable,  well  organized, 
orderly  and  planful;  use  their 
time  efficiently;  prioritize  tasks; 
stay  on  schedule;  are  not  easily 
distracted  or  bored  by  routine 
tasks;  do  not  procrastinate,  even 
when  tasks  are  unpleasant  or 
unexciting. 

Are  unreliable;  fall  behind  in 
assignments  or  duties;  miss 
deadlines;  put  off  unpleasant  tasks 
and  are  easily  distracted  while 
working  them;  rarely  plan  before 
starting  a  task;  often  lose  things. 

Dutifulness  and 

1  ntegrity  ( DUT) 

Have  a  strong  sense  of  duty  and 
moral  obligation;  try  to  do  what 
is  right  and  ethical;  accepts 
authority  and  follows  rules  and 
regulations;  fulfill  their 
obligations  and  commitments; 
accept  the  consequences  of  their 
actions. 

Are  rebellious  and  contemptuous 
of  rules  and  regulations;  refuse  to 
be  held  accountable  for  their  own 
actions;  are  undisciplined  and  self- 
indulgent;  cannot  be  trusted  and 
break  promises. 

A-i 


Table  A- 1 

NCAPS  scales  and  descriptions  of  the  attributes  of  high  and  low  scores 


Scales 

High  Scores 

Low  Scores 

Self-Reliance 

(SRL) 

Are  self-sufficient,  resourceful, 
and  like  to  make  their  own 
decisions;  avoid  becoming 
dependent  on  others  to  get 
things  done. 

Frequently  rely  on  others  to  get 
things  done;  easily  become 
dependent  on  others  for 
reassurance;  may  feel  insecure 
without  support;  often  take  up 
excessive  time  of  receptive 
listeners  seeking  support. 

Social  Orientation 
(SO) 

Like  to  work  with  others  rather 
than  alone;  like  and  accept 
people  readily;  are  sensitive  to 
others'  needs  and  feelings;  are 
understanding  and  helpful; 
increase  cohesiveness  in  groups 
in  which  they  participate. 

Are  shy,  reserved  and  aloof;  prefer 
to  be  alone;  are  insensitive  to 
others' feelings;  are  critical  and 
generally  unaccepting  of  others; 
create  friction  when  others  are 
around. 

Stress  Tolerance 
(ST) 

Maintains  composure  and  retain 
ability  to  think  clearly  and  take 
effective  action  when  confronted 
with  stressful  situations;  can 
readily  put  aside  worries  and 
feelings  of  guilt;  accept  criticism 
without  becoming  upset. 

Becomes  indecisive  or  make  poor 
decisions  in  times  of  stress  due  to 
loss  of  composure;  are  prone  to 
feelings  of  worry,  guilt,  and 
vulnerability;  are  easily  upset; 
tend  to  ruminate  about  troubling 
events  and  perceived  failures;  do 
not  take  criticism  well. 

Vigilance  (VI G) 

Are  able  to  constantly  scan  the 
environment  for  things  that 
require  attention,  even  when  no 
action  is  required  for  long  periods 
of  time. 

Over  long  periods  of  time, 
experience  lapses  in  attention 
when  required  to  scan  the 
environment  for  low  frequency  but 
critical  actions  or  events. 

Willingness  to 
Learn  (WTL) 

Demonstrates  a  willingness  to 
learn  new  material  in  classroom 
or  on  the  job  and  apply  that 
material  to  new  work  situations; 
learns  from  mistakes,  asks 
questions  when  needed  and 
takes  advice;  actively  seeks  out 
learning  opportunities. 

Avoids  training  opportunities; 
doesn't  apply  new  knowledge  to 
work  situations;  doesn't  learn  from 
mistakes  or  others  advice;  have  a 
narrow  range  of  interests. 

Table  A- 1 

NCAPS  scales  and  descriptions  of  the  attributes  of  high  and  low  scores 


Scales 

High  Scores 

Low  Scores 

Leadership 
Orientation  (LO) 

Are  willing  to  lead,  take  charge, 
offer  opinions  and  direction,  and 
take  responsibility  for  guiding 
others'  actions;  assume  the  role 
of  leader  when  no  one  else  steps 
forward;  are  able  to  mobilize 
others  to  act;  are  confident, 
forceful,  firm,  and  decisive. 

Are  unwilling  to  offer  opinions  and 
direction;  will  not  step  forward  to 
lead  when  no  one  else  will;  unable 
to  mobilize  others  into  action; 
does  not  appear  to  others  as 
decisive,  confident,  forceful,  or 
firm. 

Self-Control  (SC) 

Plans  actions  when  new 
challenges  arise;  always  think 
thorough  possible  consequences 
when  speaking;  doesn't  vent 
emotions  to  others. 

Tends  to  act  on  the  "spur  of  the 
moment;”  speaks  out  and  vents 
emotions  without  thinking  through 
possible  consequences. 

Perceptiveness 
and  Depth  of 
Thought  (PDT) 

Interested  in  pursuing  topics  in 
depth;  enjoys  abstract  thought; 
seeks  to  understand  the  "big 
picture;''  seeks  knowledge 
outside  of  own  specialty;  needs 
to  understand  how  things  work 
and  the  underlying  causes  of  a 
problem. 

Doesn't  seek  knowledge  outside  of 
personal  area  of  responsibility; 
needs  to  operate  in  concrete 
terms;  not  interested  in 
understanding  the  underlying 
causes  of  problems. 

1  nnovation  (1 NV) 

Are  able  to  come  up  with  new 
ideas  for,  and  answers  to,  work- 
related  problems;  does  not  stick 
to  old  approaches  simply  because 
things  have  always  been  done 
that  way;  looks  at  old  things  in 
new  ways;  are  open  to  new  ideas 
and  alternate  ways  of  thinking; 
are  inventive  and  imaginative. 

Likes  to  stick  to  establish  methods 
rather  than  experimenting  with 
new  approaches;  have  little  or  no 
desire  to  innovate  or  think 
creatively;  become  impatient  when 
others  seek  to  brainstorm  new 
ideas  or  approaches. 

Initiative  (INI) 

Takes  action  at  one's  own 
discretion;  willing  to  take  on 
responsibilities  and  challenges; 
persists  in  the  face  of  obstacles 
and  overcomes  barriers; 
volunteers  for  tasks  outside  of 
regular  duties. 

Unable  to  take  action  on  one's 
own;  avoids  taking  on 
responsibilities  and  challenges; 
often  gives  up  in  the  face 
obstacles,  cannot  overcome 
barriers;  rarely  volunteers  for 
tasks  outside  of  regular  duties. 

Table  A- 1 

NCAPS  scales  and  descriptions  of  the  attributes  of  high  and  low  scores 


Scales 

High  Scores 

Low  Scores 

Empathy  (EMP) 

Recognizes  and  understands 
other's  states  of  mind  or 
emotions;  demonstrates 
compassion  towards  others; 
takes  care  of  others  in  need; 
provides  sympathy  and  comfort; 
helps  others. 

Seldom  recognizes  or  understands 
other's  states  of  mind  or  emotions; 
avoids  situations  where  they  may 
need  to  comfort  others  in  need; 
fails  to  demonstrate  compassion 
toward  others;  unsympathetic 
towards  others. 

Commitment 

(COM) 

Is  psychologically  and 
emotionally  attached  to  the  Navy; 
identifies  with,  is  involved  in,  and 
enjoys  being  in  the  Navy;  views 
own  values  as  congruent  with 

Navy  values;  feels  a  sense  of 
obligation  toward  the  Navy; 
believes  that  staying  in  the  Navy 
is  the  right  thing  to  do. 

Is  not  psychologically  and 
emotionally  attached  to  the  Navy; 
doesn't  identify  with,  is  not 
involved  in,  and  doesn't  enjoy 
being  in  the  Navy;  views  own 
values  as  incongruent  with  Navy 
values;  feels  no  sense  of  obligation 
toward  the  Navy;  believes  that 
staying  in  the  Navy  is  the  wrong 
thing  to  do. 

Positive  Self- 
Concept  (PSC) 

Positively  evaluates  self; 
confident  in  own  competence  and 
capabilities;  believes  outcomes 
are  determined  by  one's  own 
behavior  rather  than  luck  or  fate; 
optimistic,  enthusiastic,  and 
cheerful. 

Negatively  evaluates  self;  not 
confident  in  own  competence  and 
capabilities;  believes  outcomes  are 
determined  by  luck  or  fate  rather 
than  one's  own  behavior; 
pessimistic,  lacks  enthusiasm,  and 
cheerless. 

Tolerance  for 
Ambiguity  (TA) 

Handles  uncertain  and 
unstructured  situations  effectively 
and  with  confidence;  prefers 
unpredictable  work  environments 
in  which  the  problems  (and 
potential  solutions)  are  ill- 
defined. 

Cannot  handle  uncertain  and 
unstructured  situations  effectively 
and  with  confidence;  prefers 
predictable  work  environments  in 
which  the  problems  (and  potential 
solutions)  are  well-defined. 
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